RFCT: Multimodal Sensing Enhances Grasping State Detection for Weak-Stiffness Targets

Accurate grasping state detection is critical to the dexterous operation of robots. Robots must use multiple modalities perceive external information, similar humans. The direct fusion method visual and tactile sensing may not provide effective visual–tactile features for network target. To address this issue, we present a novel model (i.e., RFCT) an incremental dimensional tensor product detecting states weak-stiffness targets. We investigate whether convolutional block attention mechanisms (CBAM) can enhance feature representations by selectively attending salient cues while suppressing less important information eliminating redundant initial fusion. conducted 2250 experiments using 15 used 12 targets training three testing. When evaluated on untrained targets, our RFCT achieved precision 82.89%, recall rate 82.07%, F1 score 81.65%. compared performance with various combinations Resnet50 + LSTM C3D models commonly in detection. experimental results show that significantly outperforms these models. Our proposed provides accurate has potential robust support robot operations real-world applications.